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Abstract 

In  the  ever  increasing  complexity  of  defense  acquisition,  the  traditional  metrics  for  defense 
projects  of  cost,  schedule,  and  performance  are  insufficient.  This  paper  explores  the  concept 
of  cost,  schedule,  and  performance  to  determine  if  these  three  quintessential  project 
management  criteria  are  sufficient  to  serve  as  the  guiding  principles  for  defense  project 
managers.  The  ever  increasing  complexity  of  the  weapons  system  development  environment, 
from  the  necessity  for  specialization  to  the  intricacies  of  new  technology,  requires  a  broader 
view  than  that  offered  by  cost,  schedule,  and  performance.  In  squaring  the  project 
management  circle,  we  must  add  a  fourth  variable,  context,  to  provide  focus.  Because  if  we 
measure  it,  it  will  get  done. 

Introduction 

This  paper  is  an  examination  of  the  complexity  of  defense  acquisition  and  its 
relationship  to  the  measures  of  cost,  schedule,  and  performance — the  project  management 
circle  (see  Figure  1 ).  Rather  than  use  the  more  traditional  name  of  project  management 
triangle  (or  triple  constraint,  or  even  the  iron  triangle),  we  refer  to  the  concept  as  the  project 
management  circle.  The  circle  recognizes  the  interrelationships,  necessary  equilibrium,  and 
the  influencing  and  balancing  effects  that  these  three  variables  provide.  This  paper  seeks  to 
demonstrate  that  these  three  variables  are  missing  an  important  consideration  that  should 
be  held  in  equal  regard.  Recognizing  that  fourth  consideration  will  allow  the  circle  to  be 
squared. 


Figure  1.  The  Project  Management  Circle 

Any  discussion  of  cost,  schedule,  and  performance  must  include  the  concept  of 
project  success  as  ultimately,  cost,  schedule,  and  performance  are  meant  to  ensure 
success.  These  three  ideas  form  the  heart  of  the  concept  of  project  management  and  are 
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seen  as  not  only  management  concepts,  but  as  the  definition  of  success  in  project 
management.  In  fact,  cost,  schedule,  and  performance  are  the  prevailing  criteria  used  to 
evaluate  project  success  in  the  U.S.  Department  of  Defense,  and  indeed  throughout  the 
U.S.  Government.  Congress  uses  cost,  schedule,  and  performance;  the  media  tend  to  focus 
on  cost,  schedule,  and  performance;  and  the  GAO  almost  exclusively  uses  cost,  schedule, 
and  performance  to  measure  execution.  This  paper  suggests  that  the  cost,  schedule,  and 
performance  paradigm,  while  still  effective  as  a  measure  of  managing  programs,  needs  to 
be  expanded  or  changed. 

The  Problem 

The  current  practice  of  project  management  assumes  a  simple,  structured,  and 
stable  environment  where  the  basic  ideas  of  cost,  schedule,  and  performance  are  sufficient 
to  capture  the  workings  of  weapons  system  development,  as  well  as  serving  to  define 
success.  However,  the  DoD  environment  is  complex,  dynamic,  and  constantly  changing, 
and  defining  success  is  problematic.  In  this  21st  Century  environment,  the  obligation  of 
management  in  general  and  that  of  project  managers  specifically  is  to  deal  with  complexity. 
Getting  the  system  developed  and  fielded,  regardless  of  complexity,  is  the  focus  of  the 
project  management  effort. 

Simultaneously,  while  cost  and  schedule  measures  remain  important,  they  are 
insufficient  as  measures  of  project  success.  Nevertheless,  the  management  concepts  of 
cost,  schedule,  and  performance  remain  the  same.  We  continue  to  manage  and  define 
success  in  acquisition  using  cost,  schedule,  and  performance.  While  important  measures, 
cost  schedule,  and  performance  are  insufficient  criteria  for  both  program  management  and 
program  success.  This  is  the  dilemma  facing  project  managers  today. 

The  Approach 

The  research  seeks  to  examine  the  complexity  of  defense  project  management  and 
relate  that  complexity  to  the  key  variables  of  cost,  schedule,  and  performance.  The  intent  is 
to  explore  other  variables  that  will  better  help  to  explain  DoD  project  success  (or  failure)  and 
provide  the  DoD  weapons  system  project  manager  the  ability  to  manage  more  effectively. 
This  research  attempts  to  “square”  the  project  management  circle  by  identifying  a  fourth 
critical  variable  that  must  be  addressed  by  project  managers.  The  research  methodology 
consists  of  a  system-focused  approach  based  on  an  extensive  review  of  the  literature  of 
cost,  schedule,  and  performance,  and  project  complexity. 

The  analysis  consists  of  three  parts.  The  first  section  will  examine  the  concept  of 
cost,  schedule,  and  performance.  The  second  section  explores  DoD  project  management 
complexity.  Project  success  is  also  examined  as  it  relates  to  complexity  and  defense 
acquisition. 

Lastly,  a  fourth  variable,  the  concept  of  project  management  context,  is  introduced. 
We  explore  the  idea  of  project  context,  identifying  and  categorizing  the  context  of  the 
defense  project  using  a  systems  framework.  The  results  of  the  analysis  will  identify  those 
context  variables  that  would  contribute  to  a  project  management  model  addressing  complex 
weapons  systems  development.  The  expected  result  is  the  identification  of  variables, 
beyond  cost,  schedule,  and  performance,  that  contribute  to  project  success  and  enable  the 
complex  systems  project  manager  to  better  address  the  management  challenges  evident  in 
most  DoD  systems  development. 
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Cost,  Schedule  &  Performance 

Cost,  schedule,  and  a  third  measure — performance,  scope,  or  quality,  among 
others — are  as  old  as  the  practice  of  project  management.  Project  managers  apply 
management  principles  and  knowledge  to  effect  change.  The  Project  Management  Institute 
(PMI;  n.d.)  defines  a  project  as  “a  temporary  endeavor  undertaken  to  create  a  unique 
product,  service  or  result.”  The  temporary  aspect  emphasizes  the  limits  of  project 
management,  the  application  of  finite  resources — both  time  (schedule)  and  budget  (cost). 
The  unique  property  underlines  the  focus  and  purpose  of  the  project,  as  well  as  the  result — 
performance.  No  one  would  dispute  the  importance  of  these  concepts,  but  are  these  criteria 
sufficient  in  today’s  complex  environment?  The  reality  is  projects  in  general,  and  complex 
projects  in  particular  (including  defense  projects),  are  often  completed  late,  over  budget  or 
both  (Morris  &  Hough,  1988). 

While  well  known  to  the  practice  of  project  management,  cost,  schedule,  and 
performance  are  elusive  concepts  in  the  context  of  the  academic  literature.  Cost,  schedule, 
and  scope  in  the  construction  industry  are  prevalent,  but  cost,  schedule,  and  performance 
as  used  in  defense  are  not.  Most  research  on  cost,  schedule,  and  performance  generally 
focuses  on  the  specifics  of  earned  value  and  the  mechanics  of  managing  weapons  systems 
projects  (Atkinson,  Crawford,  &  Ward,  2006;  Aubry,  Hobbs,  &  Thuillier,  2007;  Gardiner  & 
Stewart,  2000). 

The  business  of  defense  project  management  is  to  create  a  product — a  system 
comprised  of  advanced  technology  for  the  most  part,  a  weapons  system  (Gaddis,  1959). 
Defense  project  managers  manage  work  (or  scope),  technology,  people,  interfaces  and  the 
overall  system  to  ensure  a  viable  result.  Cost,  schedule,  and  performance  are  metaphors  for 
trade-offs.  The  project  manager  must  manage  many  obvious  and  sometimes  not  so  obvious 
constraints  and  trade  those  constraints  against  each  other  (Caccamese  &  Bragantini,  2012). 
This  decision  function  is  the  essence  of  project  management. 

Systems  take  inputs  and  transform  those  inputs  into  outputs.  Mastery  of  project 
management  requires  recognition  that  the  project  is  a  system  and  that  the  “black  box” 
(transformation)  process  of  systems  development  sometimes  produces  unforeseen  results 
(outputs).  These  unforeseen  results  include  a  continuum  that  ranges  from  success  to  failure 
in  both  capability  and  management.  Figure  2  is  an  Integrated  Definition  for  Function 
Modeling  (IDEF0)  representation  of  the  system.  Inputs  are  combined  with  resources  to 
produce  outputs.  Controls  are  the  constraints  of  the  system;  budget  and  time  are  most  often 
the  primary  constraints,  with  performance  both  a  constraint  as  well  as  an  output  of  the 
system.  The  mechanisms  include  the  actual  work  and  application  of  skills  to  transform  the 
system.  Management  of  the  diverse  factors  that  form  the  process  of  weapons  system 
development  results  in  functional  integration.  Systems  integration  further  highlights  the 
systems  nature  of  defense  project  management. 
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Figure  2.  The  Systems  Nature  of  Project  Management 

We  manage  this  system  by  monitoring,  measuring  and  attempting  to  control  cost, 
schedule,  and  performance.  However,  we  often  fail  to  meet  these  measures.  Could  it  be  that 
DoD  projects  continue  to  fail  because  our  management  tools  consist  of  this  limited  set  of 
measurement  criteria?  After  all,  the  old  phrase  attributed  to  past  systems  thinkers  and 
management  experts,  “what  gets  measured  gets  managed,”  is  as  actual  today  as  when  it 
was  first  voiced  (Willcocks  &  Lester,  1996).  Using  the  same  thinking,  a  corresponding 
phrase,  “what  gets  measured  gets  done,”  also  rings  weak  in  that  we  are  certainly 
measuring,  but  in  many  cases,  project  success  isn’t  getting  done. 

Further,  even  if  the  cost,  schedule,  and  performance  criteria  are  achieved,  the  only 
thing  demonstrated  is  that  we  are  meeting  goals  rather  than  accurately  measuring  success 
(Atkinson,  1999).  Are  meeting  these  criteria — cost,  schedule,  and  performance — critical  for 
success,  without  which  the  weapons  system  development  is  classified  as  a  failure?  Senior 
DoD  officials  routinely  point  out  that  notwithstanding  cost  and  schedule  overruns,  the 
weapons  systems  the  DoD  produces  are  the  best  in  the  world. 

Cost,  schedule,  and  performance  were  sufficient  for  the  management  of  simple 
programs  and,  in  some  cases,  complicated  programs.  Complicated  programs,  while 
sometimes  large  and  consisting  of  many  moving  parts,  operate  in  predictable  ways  (Sargut 
&  McGrath,  2011).  The  operation  of  a  weapons  system,  while  complicated  (and  difficult),  is 
predictable.  Complex  programs  are  different.  While  complex  systems  may  operate  in 
predictable  ways,  the  interactions  of  elements  of  the  complex  system  are  unpredictable,  as 
they  constantly  change  (Sargut  &  McGrath,  2011).  By  their  very  nature,  cost,  schedule,  and 
performance  are  metrics,  and  as  metrics,  become  predictors.  However,  it  is  almost 
impossible  to  predict  with  accuracy  the  end  state  of  cost,  schedule,  and  performance  in 
complex  systems. 

Complexity 

Complexity  is  the  major  dynamic  of  weapons  system  development  in  the  21st 
Century.  Complexity  is  ever-present,  but  at  the  same  time  constantly  changing.  The 
continued  growth  of  complexity  has  changed  the  process  and  the  organizations  of  project 
management  in  important  ways.  Managerial  and  technical  complexity,  and  the  resultant 
recognition  of  the  limits  of  human  capability,  has  resulted  in  necessary  changes  in  both 
human  and  organizational  capacity.  From  the  human  perspective,  complexity  has  spawned 
specialists — experts  in  a  particular  field — able  to  address  those  smaller  aspects  of  a 
complex  system  that  can  be  handled  by  a  single  person.  The  need  to  deal  with  technical 
complexity  while  ensuring  system  capability  is  the  basis  for  the  field  of  systems  engineering, 
among  other  technical  specializations. 
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Specialization  has  a  limiting  function,  in  that  the  specialists  in  a  project  organization 
are  measured  by,  and  capable  of  addressing,  only  those  issues  in  their  specific  area.  As  a 
result,  the  project  management  offices  have  increased  in  size  to  meet  the  needs  of 
specialization.  This  has  resulted  in  a  corresponding  decrease  in  the  visibility  over  the  entire 
project,  a  “can’t  see  the  forest  for  the  trees”  analogy  from  the  individuals’  perspective.  This 
has  the  potential  of  causing  a  potential  decrease  in  efficiency  in  the  execution  of  the  project. 

Complexity  in  project  management  refers  to  those  organizational,  informational  and 
technical  characteristics  of  the  project  and,  by  extension,  the  project  management 
organization  and  the  technical  staff  (Baccarini,  1996).  Included  in  the  organizational 
construct  are  the  categories  of  stakeholders  and  other  interested  parties.  Complexity  has  a 
direct  effect  on  management  and  decisions  as  the  more  complex  the  system,  the  potentially 
more  complex  the  management  effort  and  decisions  required.  The  mixture  of  human-socio¬ 
political  complexity  found  in  weapons  systems  development  offices  further  adds  to  this 
complexity  (Atkinson,  1999;  Pinto,  2000).  Finally,  complexity  reduces  the  predictability  of  the 
outcome  of  decisions  made  (Sargut  &  McGrath,  2011). 

Definitions  and  explanations  of  complexity,  managerial,  engineering  and 
technological  abound,  from  Williams  to  Gell-Mann,  to  Holland,  to  Hughes  (Gell-Mann,  1995; 
Holland,  1993;  Hughes,  1998;  Sargut  &  McGrath,  2011;  Williams,  2002).  From  the  project 
management  perspective,  Baccarini  (1996)  identifies  two  elements  of  complexity, 
organizational  and  technological  complexity.  He  further  subdivides  these  functions  into 
differentiation  and  interdependency.  Differentiation  refers  to  the  varied  size  and  structure  of 
projects  and  the  organizations  that  manage  them,  while  interdependency  describes  the 
activities  between  these  varied  elements  (Baccarini,  1996). 

Williams  builds  on  the  Baccarini  topology  and  defines  project  complexity  as 
categories  in  two  key  areas,  structural  complexity  and  uncertainty  (Williams,  2005). 
Structural  complexity  is  a  result  of  the  number  of  elements  of  a  project,  the  pieces,  including 
the  people,  the  organizations,  and  the  technology,  coupled  with  the  way  these  pieces 
interact,  their  interdependencies.  This  combination  of  interactions  of  the  varied  elements  is 
structural  complexity  (Williams,  2002).  Williams’  (2002)  second  aspect  of  complexity  is 
uncertainty  of  the  goals  and  the  methods  necessary  to  reach  those  goals. 
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Table  1.  Project  Management  Complexity 

(Baccarini,  1996;  Sargut  &  McGrath,  2011;  Sheard  &  Mostashari,  2009;  Williams,  2002) 


Type 

Sub-type 

Acquisition  Management  Example 

Size 

Organization  (number  of  people) 

Budget 

Scope  of  work 

Contractor  (size  and  number  of  people) 

Structural 

Connectivity/ 

Actions/ 

Approvals 

Acquisition  organizations 

Requirements  organizations 

Industry  organization 

Review  processes  (both  programmatic  and  technical) 

Organizational 

Stakeholder  Organizations 

Boundaries/  different  commands/  different  agencies 
Executive  Branch 

Congress _ 

Budget 

Funding 

Uncertainty 

Technical 

Complexity 

Variety  of  tasks 

Interdependencies  between  tasks 

Objectives 

System  Requirements 

Dynamic 

Short-term 

Daily  problems 

Personnel  changeover 

Engineer  shortage 

Materials  failures 

Short  requirement  dynamics 

Rework 

Long-Term 

Changing  budget 

Environment 

Socio-Political 

Social-Political 

Personnel  changeover 
“the  new  PEO/  PM” 

Change  and  change  management 

Regulations/  Policy  changes 

System 

Interdependency 

Emergence 

Unanticipated  actions  and  consequences  a  result  of 
incomplete  appreciation  of  system 

Sargut  and  McGrath  (2011)  identify  three  properties,  multiplicity,  interdependence 
and  diversity,  as  key.  Multiplicity  refers  to  the  number  of  interacting  elements  or  scale.  This 
is  similar  to  the  Williams  construct  of  structural  complexity.  Interdependence  is  the 
connectivity  of  different  elements.  And  diversity  is  a  measure  of  the  difference  in  the 
elements  (Sargut  &  McGrath,  201 1 ). 

From  the  systems  side,  Sheard  and  Mostashari  (2009)  explain  project  complexity 
from  the  systems  engineering  perspective.  That  view  acknowledges  structural  complexity, 
but  adds  dynamic  and  socio-political  complexity  as  factors  influencing  complex  systems 
development.  Dynamic  complexity  recognizes  the  active  nature,  the  change-over-time,  of 
systems  development.  Socio-political  complexity  reflects  the  human  side  of  complexity, 
focusing  on  the  importance  and  challenges  of  social  interaction  in  systems  development, 
including  the  cognitive  challenge  complexity  causes,  as  well  as  the  effect  of  everyday 
politics. 

To  allow  for  a  more  complete  analysis,  the  complexity  frameworks  developed  by 
Williams  based  on  project  management,  by  Sheard  and  Mostashari  based  on  systems 
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engineering,  and  discussed  by  Sargut  and  McGrath  based  on  business  considerations  are 
combined  to  illustrate  project  management  complexity  in  the  Department  of  Defense 
(Baccarini,  1996;  Williams,  2002;  Sargut  &  McGrath,  2011;  Sheard  &  Mostashari,  2009). 
While  the  Sheard  and  Mostashari  (2009)  framework  is  focused  on  systems  engineering,  it  is 
valuable  because  it  provides  an  important  link  from  engineering  to  project  management.  The 
resulting  framework  (Table  1 )  includes  a  topology  of  different  kinds  of  structural  complexity, 
uncertainty,  dynamic  and  socio-political  complexity,  and  overall  system  complexity. 

This  grouping  of  complexity  factors  combines  the  management  and  engineering 
considerations  in  the  development  of  weapons  systems.  In  many  cases,  the  resultant 
complexity  is  manifested  in  more  than  one  type.  For  example,  personnel  issues  including 
leadership  changes  have  complexity  effects  in  uncertainty,  dynamic  complexity,  and  socio¬ 
political  complexity. 

Structural  complexity  includes  the  scale,  connectivity,  organizational  structure,  and 
objectives  of  the  development.  Size  is  about  magnitude  of  the  acquisition  system  and  its 
policies,  bureaucracy  and  hierarchy  to  include  the  private  sector  side  of  defense  acquisition. 
Connectivity  acknowledges  that  the  volume  of  staff  actions  between  these  organizations  is 
significant  and  includes  both  issues  relating  to  managing  ongoing  development.  The 
connectivity  aspect  of  structural  complexity  is  influenced  by  the  nature  of  defense  acquisition 
systems.  Since  the  technology  development  infrastructure  (i.e.,  laboratories,  R&D  centers, 
and  manufacturing)  is  for  the  most  part  privately  owned,  structural  complexity  also  describes 
the  network  connectivity  necessary  for  the  system  to  function.  Beyond  the  hierarchies, 
project  organizations  are  major  business  entities  directly  controlling  budgeting,  spending 
and,  in  most  cases,  the  award  of  fee  to  defense  companies.  Project  organizations  are 
spread  throughout  the  United  States  and  overseas,  further  adding  to  the  complexity.  Finally, 
the  focus  on  defense  project  management  by  the  DoD  essentially  means  project 
organizations  extend  from  the  task  level  of  the  project  to  the  highest  levels  of  the 
bureaucracy.  A  recent  GAO  study  recognized  the  challenges  of  structural  complexity  in 
finding  the  reviews  for  some  programs  include  up  to  56  organizations  at  eight  levels.  These 
structural  requirements,  reviews  and  responding  to  information  requests  can  add  up  to  two 
years  to  the  development  time  (GAO,  2015). 

Uncertainty  focuses  on  three  major  areas:  budget,  technical  complexity,  and  overall 
system  objectives.  In  defense  acquisition,  budget  is  a  major  concern  and  source  of 
uncertainty  because  of  the  year-to-year  budget  cycle,  as  well  as  political  considerations. 
Uncertainty  also  stems  from  the  military  rotation  policy,  where  senior  leaders  change  jobs 
approximately  every  two  to  three  years.  Most  new  leaders  are  driven  to  make  a  mark  on  the 
organization  and  may  be  therefore  unwittingly  contributing  to  the  uncertainty  of  the  staff. 
Technical  complexity  is  a  fact  of  life  in  defense  systems,  and  the  reality  is  while  we  plan  for 
technological  development,  it  is  in  fact  an  estimate  only.  As  we  develop  systems,  we  learn 
more  about  the  technologies,  and  are  then  better  able  to  plan  for  schedule,  and  cost. 

Dynamic  complexity  is  classified  as  short  and  long  term  and  generally  refers  to 
change  and  time  available.  This  concept  is  divided  into  the  short  and  the  long  term  because 
of  the  differences  in  perspective,  as  well  as  the  universe  of  potential  reactions  to  dynamic 
complexity  (Sheard  &  Mostashari,  2009).  Whether  it  is  a  tactical  response  to  a  development 
problem  or  an  administrative  response  to  directives,  the  project  management  system  is  in 
constant  flux.  This  dynamic  is  a  function  of  the  diverse  and  always  changing  aspects  of 
ongoing  development.  Further,  each  individual  (the  human  element)  will  interpret  and 
emphasize  different  aspects  of  the  problem  and  how  to  address  that  problem.  This  has  a 
potentially  significant  impact  on  the  management  system. 
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Socio-political  complexity  is  the  nexus  between  management,  and  the  non¬ 
engineering  human  factors  of  policy,  process  and  practice  of  the  system  is  most  critical 
(Maier,  1995).  Socio-political  complexity  also  recognizes  the  politics  of  project  management, 
starting  with  the  budget  process,  through  Congress,  and  back  into  the  development 
organizations.  An  oft  overlooked,  but  critical  aspect  of  context  is  politics.  In  fact,  politics  is  by 
far  the  most  powerful  factor  in  the  category  of  context.  Most  engineers  and  project 
managers  dismiss  politics  as  the  realm  of  higher-level  decision  makers.  In  fact,  many  refuse 
to  engage  in  politics  as  they  find  the  practice  distasteful  (Pinto,  2000).  However,  dismissing 
those  political  activities  can  have  consequences.  Whenever  people  are  put  in  an 
organization  and  asked  to  function  as  a  team,  there  is  an  inevitable  use  of  power  and 
political  behavior  (Pinto,  2000).  Notwithstanding  a  general  distaste  for  political  behavior  in 
the  workplace,  the  reality  is  the  practice  of  politics  is  a  prime  force  in  any  weapons 
development. 

The  last  aspect  of  complexity  in  the  context  of  program  management  is  overall 
system  complexity.  System  complexity  is  the  result  of  the  interaction  of  all  the  stated 
elements  of  project  complexity.  When  different  systems  interact,  or  when  different  aspects  of 
complexity  act  on  each  other,  there  are  two  results.  The  first  is  the  cumulative  effect  of  the 
interaction.  For  the  project  organization,  the  interdependencies  between  those  managing  the 
development  and  those  executing  the  development  should  result  in  repeatable,  consistent 
results — continued  progress  in  system  development  (Rebovich,  2008).  However,  when  the 
link  between  those  managing  and  those  executing  is  broken  or,  as  can  happen,  ignored,  the 
interdependency  is  broken. 

In  today’s  environment,  the  concentration  of  the  defense  project  manager  role  on  the 
program  (work/scope),  the  technology  to  be  developed,  the  interfaces  of  the  system 
including  those  non-technical  interfaces,  and  the  project  environment — the  ecosystem  of  the 
development.  Thus,  complexity  requires  a  different  approach  to  project  management,  one 
that  acknowledges  the  importance  of  managing  resources  (cost  and  schedule)  to  optimize 
system  performance,  but  at  the  same  recognizes  the  crucial  known  and  unknown 
constraints  and  interdependencies  of  the  environment — the  context  of  the  system 
development. 

Project  Success 

Cost,  schedule,  and  performance  are  both  a  management  tool  as  well  as  a  predictor 
of  success.  The  management  science  discipline  has  sought  to  quantify  the  activities  of  the 
various  management  disciplines,  including  project  management.  Tishler  et  al.  (1996) 
observed  that  in  order  to  identify  the  managerial  factors  (and  by  extension  the  processes 
leading  to  those  factors),  success  must  be  defined.  They  further  cite  research  by  Pinto  and 
Slevin  (1998)  that  definitions  of  success  change  during  different  phases  of  the  lifecycle. 

A  major  focus  of  the  literature  on  project  success  has  been  on  the  idea  of  success 
criteria,  or  critical  success  factors  (CSF;  Jugdev  &  Muller,  2005).  Identified  success  factors 
include  cost,  schedule,  and  performance,  as  well  as  project  functionality  and  its 
management  (Morris  &  Hough,  1988).  Further  studies  added  criteria  such  as  customer 
satisfaction,  efficiency  of  execution,  and  effectiveness  of  the  project  organization  (Pinto  & 
Slevin,  1998).  Tishler  et  al.  (1996)  suggest  groupings  of  four  critical  success  factors  for 
defense  oriented  projects,  preparation,  quality  of  the  system  development  team  and  the  user 
customer  organization,  management  policy  and  project  control.  This  clustering  of  success 
factors  represents  the  amalgamation  of  the  broader  literature  on  project  success  criteria 
(Atkinson,  1999;  Cooke-Davies,  2002;  De  Wit,  1988;  Morris  &  Hough,  1988;  Pinto  &  Mantel, 
1990;  Pinto  &  Slevin,  1998). 
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Preparation  for  project  execution  includes  the  necessary  planning  for  initiating  the 
project,  as  well  as  the  necessary  coordination.  Included  in  the  idea  of  preparation  is  an 
assessment  of  the  urgency  of  need  as  urgency  in  defense  projects  overcomes  many 
constraints.  Team  quality  refers  to  both  the  management  as  well  as  technical  capabilities  of 
the  development  organization.  Management  policy  is  focused  on  quality,  producibility,  and 
design-to-cost  considerations.  Project  control  relates  to  the  systematic  use  of  control 
methods  for  cost,  schedule,  and  performance.  Together,  these  factors  represent  the  criteria 
generally  necessary  for  projects  to  be  successful.  This  suggests  that  the  exclusive  and  rigid 
adherence  to  cost,  schedule,  and  performance  as  indicators  of  success  (and  the  hallmark  of 
defense  project  management)  alone  does  not  reflect  the  totality  of  success  in  defense 
project  management. 

Most  importantly,  the  literature  identifies  two  kinds  of  success,  project  success  and 
project  management  success  (Cooke-Davies,  2002;  De  Wit,  1988;  Jugdev  &  Muller,  2005). 
Project  success  is  measured  as  achieving  technical  performance  and/or  mission 
performance  goals,  coupled  with  customer  (warfighter)  satisfaction  (De  Wit,  1988).  Project 
success  is  measured  against  the  overall  objectives  of  the  development  (Cooke-Davies, 
2002).  The  nature  of  defense  acquisition  requires  weapons  systems  that  function  as 
intended.  This  is  measured  in  very  real  terms  of  life  or  death  and  battlefield  success  or 
failure.  Rigid  adherence  to  and  sole  focus  on  cost,  schedule,  and  performance  mean  little  if 
the  system  does  not  perform  when  needed  (Cleland  &  King,  1983).  In  the  greater  scheme  of 
things,  what  matters  in  defense  acquisition  is  whether  the  system  functions  as  the  warfighter 
needs. 


A  major,  complex  project’s  principal  success  criteria  will  vary  over  time  (Atkinson, 
1999;  De  Wit,  1988;  Morris  &  Hough,  1988).  To  paraphrase  de  Wit  (1988),  defense  projects 
have  at  least  three  specific  indicators  of  success:  identifying  the  technology,  developing  the 
technology,  and  developing  the  weapons  system  using  the  technology.  Delivering  capable 
weapons  systems  is  project  success.  But,  identifying  the  technology  and  developing  the 
technology  are  also  measures  of  project  success. 

Cost,  schedule,  and  performance  measure  project  management  success.  At  its 
heart,  project  management  success  is  a  measure  of  how  efficiently  the  project  has  been 
managed  (Baccarini,  1999).  Project  success  is  different  from  project  management  success. 
Project  success  will  be  determined  by  the  warfighter  community. 

Project  management  success  includes  overcoming  issues  such  as  supply-chain 
challenges  and  effective  coordination  within  the  project  management  office.  Project 
management  success  is  focused  on  the  development  process.  The  DoD,  the  GAO,  and 
Congress  measure  project  management  success,  rather  than  project  success.  Project 
management  success,  however,  is  and  must  remain  subordinate  to  project  success.  Cost, 
schedule,  and  performance  are  inadequate  indicators  even  for  project  management 
success. 

Context,  the  Fourth  Criterion 

The  factors  of  complexity  identified  in  Table  1  provide  a  starting  point  for 
identification  of  factors  that  are  beyond  the  basics  of  cost,  schedule,  and  performance,  yet 
influence  project  and  project  management  success.  From  a  systems  perspective,  execution 
of  weapons  system  development  must  be  considered  from  the  viewpoint  of  all  stakeholders 
(Owens  et  al.,  201 1 ).  This  viewpoint  includes  an  appreciation  of  the  identified  elements  of 
complexity,  including  structure,  uncertainty,  dynamics,  socio-political  and  system.  These 
factors  of  complexity  constantly  shape  the  project  environment  and  form  the  basis  of  the 
context  of  the  system  within  which  the  project  manager  must  operate.  We  group  these 
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complexity  factors  that  are  the  result  of  the  execution  of  a  project  together  and  define  them 
as  context  (Owens  et  al.,  201 1 ). 

Context  includes  those  project  organization  activities  that  are  essential  to  administer 
programs,  but  are  not  directly  related  to  the  cost,  schedule,  and  performance  of  the  project. 
Context  ranges  from  tracking  budget  requests  through  the  bureaucracy  to  responding  to 
stakeholder  inquiries  on  how  resources  are  being  used.  In  weapons  system  development, 
context  includes  those  activities  that,  while  not  tied  directly  to  cost,  schedule,  and 
performance,  are  essential  for  execution. 

Each  project  is  unique,  a  mix  of  many  factors.  More  than  cost,  schedule,  and 
performance,  context  reflects  the  ecosystem  of  the  project  organization  and  the  project.  If 
cost,  schedule,  and  performance  are  measures  and  criteria  for  project  management 
success,  context  is  a  criterion  for  project  success. 

The  goal  of  this  paper  was  to  explore  the  concept  of  cost,  schedule,  and 
performance  to  determine  if  these  three  quintessential  project  management  criteria  were 
sufficient  to  serve  as  the  guiding  principles  for  defense  project  managers.  The  ever 
increasing  complexity  of  the  weapons  system  development  environment,  from  the  necessity 
for  specialization  to  the  intricacies  of  new  technology,  requires  a  broader  view  than  that 
offered  by  cost,  schedule,  and  performance.  In  squaring  the  project  management  circle,  we 
must  add  as  fourth  variable,  context,  to  provide  focus  because  if  we  measure  it,  it  will  get 
done. 
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OVERVIEW 


•  Problems 

The  Project  Management  Circle 
Cost,  Schedule  Performance 

•  Project  Success 

Methodology — Systems  Approach 
Cost,  schedule  and  performance 

•  Defense  Project  Complexity 
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Goal:  Identify  Factors  beyond  Cost,  Schedule  &  Performance  that 
influence  defense  development  programs 


Ultimate  use  is  to  model  the  defense  acquisition  process 
to  provide  a  training  environment  for  PMs 
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PROBLEM  I— DEFENSE  SYSTEMS 
DEVELOPMENT  ENVIRONMENT 


•  Defense  systems  acquisition  has  three  major  characteristics... 

•  Complex  (well  beyond  complicated) 

•  Not  transparent  (opaque) 

External  and  internal  dynamics  that  are  not  completely  understood  by  the 
people  charged  with  their  execution. 

Results  in  imperfect  decision  making 

DoD  Focus  driven  by  cost,  schedule  and  performance  considerations 

overly  simplistic  short-term  decisions  made  without  considering  their  later 
effects....  Providing  a  less  than  optimal  decision  focus 


THINGS  PROJECT  MANAGERS 
PEOPLE  DON’T  DO  WELL 


•  Take  the  long-view... anticipating  the  consequences  of 
decisions 

•  Anticipate  future  problems 

•  Appreciate  the  effects  of  Time  (delayed  feedback  systems) 

•  Easily  understand  large  amounts  of  data 

•  Change — we  tend  to  hold  on  to  beliefs  more  strongly  when  we 
feel  insecure/challenged/  are  wrong. 

•  Deal  with  Complexity 


PROBLEM  2— DEFINING  SUCCESS 


...achieving  technical  performance 
and/  or  mission  performance  goals, 
coupled  with  customer  (warfighter) 
satisfaction 


Success 

...how  efficiently  the 
project  has  been 
managed 


Imperfect  understanding  of  project  success 
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CONCLUSION 


Cost,  Schedule  and  Performance  are  insufficient  to  inform 
today’s  defense  project/  engineering  management 
environment 

•  Context  shapes  the  project  development  process — it  is  at 
the  heart  of  successful  execution 

•  A  thorough  appreciation  of  the  effects  of  context  variables 
is  critical  for  successful  program  execution 


QUESTIONS 


•  Define  how  the  work  of  the  organization  is  done 

Logical  organization  of  people,  materials,  energy,  equipment  and 
procedures  into  work  activities  designed  to  produce  a  result. 

•  Set  of  processes  lead  to  the  accomplishment  of  a  task 

•  Cross  organizational  boundaries  (between  tasJcs^j^jiat^anizations) 

•  Process  Entities 


Interorganizational 
Interfunctional 
Interpersonal 
Process  Activities 
Operational 
Managerial 


A  Systems  Approach  to 
Project  Management 
Research  is  Essential 
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